1. Field of the Invention
The present invention relates to an image capturing apparatus, a control method therefor, and a recording medium and, more particularly, to a luminance correction technique for successively captured images.
2. Description of the Related Art
An image capturing apparatus such as a digital camera or digital video camera can correct a captured and obtained image to a desired state by applying various image processes to the image. Correction processing applied in the image capturing apparatus includes, for example, brightness correction, tone correction, and contrast correction. Most image capturing apparatuses determine the features of an obtained image, and apply those correction processes automatically or according to a set shooting mode or the like.
Correction processing for an image is applied to, for example, a backlit scene in which a light source exists in the background of an object. In a backlit scene in which the sun is in the background of a person as a main object, for example, the surface of the main object which faces the image capturing apparatus is shaded, thereby decreasing the luminance. In this case, by emitting flash light to shoot an image under exposure conditions where no blown-out highlight occurs in the background, it is possible to obtain an image in a state in which the object and the bright background are preferable while ensuring the luminance of the main object. Using flash light as artificial light may make the luminance of the main object unnatural, or cause shadow-detail loss because the flash light does not reach the main object, depending on the distance between the main object and the image capturing apparatus to decrease the luminance of the main object. In recent years, therefore, a shadow-detail loss region and a blown-out highlight region are reduced by applying, to an image shot in a backlit scene, image processing for performing tone connection for a luminance signal. More specifically, a signal amplification ratio is changed according to the signal level (brightness) of each pixel or a tone level to be assigned is adjusted for each signal level range to which each pixel belongs, thereby performing correction.
If luminance correction is performed for all the pixels of an image as described above, the following problems may occur depending on an object in the image. FIG. 9A shows an image 901 captured in a backlit scene in which the sky and a tree are included in a shooting range. In the image 901, a region corresponding to the sky has a luminance higher than that of a region corresponding to the tree, and luminance gradation in which the luminance gradually decreases from the upper portion of the image to the lower portion is seen. If luminance correction of assigning a luminance to each signal level range as described above is performed for all the pixels of the image, an image 902 shown in FIG. 9B is obtained. In the image 902, a boundary portion between the sky region with a high luminance and the tree region with a low luminance blurs. This is because the surrounding region of the tree is a so-called high-frequency component region, where pixels representing branches and leaves of the tree with a low luminance are frequently mixed with pixels representing the sky with a high luminance. In extracting a desired high-frequency component region, whether each pixel existing near the boundary between bright and dark regions is a luminance correction target pixel or non-target pixel is based, in part, on the region extraction performance. If a pixel with a high luminance level to some extent which should not actually be a correction target is extracted as a high-frequency component region to be corrected, and is corrected to be brighter, the boundary portion may blur.
A difference between correction results which have been obtained by performing luminance correction for a low-frequency component region and a high-frequency component region, respectively, will be described with reference to FIGS. 10A to 10C. FIG. 10A shows an image 1000 having a low-frequency component as a whole, and an image 1010 having a high-frequency component as a whole. The images 1000 and 1010 are formed by high-luminance regions 1001 and 1011 with a median of 180 of the luminance distribution, and low-luminance regions 1002 and 1012 with a median of 30 of the luminance distribution, respectively. In the image 1000, there is one low-luminance region. To the contrary, in the image 1010, there are a plurality of low-luminance regions, and there is a high-luminance region between the low-luminance regions. Thus, the image 1010 has a high frequency component. In this case, the number of pixels of the low-luminance region 1002 of the image 1000 is equal to the total number of pixels of the low-luminance regions 1012 of the image 1010, and the luminance distributions of the images are equal to each other, as shown in FIG. 10B.
If luminance correction for increasing the luminance of the low-luminance region is applied to the images 1000 and 1010, changes in luminance distributions of the images after the processing are different from each other, as shown in FIG. 10C. More specifically, the median of the luminance distribution of the low-luminance region 1002 of the image 1000 increases to 60, while that of the luminance distribution of the low-luminance regions 1012 of the image 1010 increases to only 40. That is, the effect of luminance correction for the low-luminance region changes depending on whether the surrounding region of the target region is a low-frequency component region or a high-frequency component region, as shown in FIG. 11. FIG. 11 shows that a correction amount after actual correction for the high-frequency component region is smaller than that for the low-frequency component region for a given luminance correction target amount; that is, only a small correction effect is produced for the former region.
On the other hand, in the image 902 of FIG. 9B, since the sky region other the surrounding region of the tree is a low-frequency component region where the luminance gradually changes, a problem that, for example, the edges blur does not arise. As shown in the image 902, however, it becomes impossible to reproduce the luminance gradation by assigning a luminance to each signal level range. Especially in a region with a high luminance, a luminance difference caused by tone assignment readily stands out, thereby giving the user the impression that degradation in tone has occurred in the corrected image.
That is, since luminance correction for a high-frequency component region of the image reduces the resolution of the image, it is preferable not to apply luminance correction to the region, or to limit the correction amount of luminance correction. Furthermore, since luminance correction for a low-frequency component region with a high luminance level of the image may give the user the impression that degradation in tone of the image has occurred, it is preferable not to apply luminance correction to the region, or limit a correction amount. Thus, Japanese Patent Laid-Open No. 2008-072604 proposes a method of applying luminance correction of a low-luminance region in a backlit scene by limiting it to a low-frequency component region.
To perform luminance correction for a low-frequency component region with a low luminance level as described in Japanese Patent Laid-Open No. 2008-072604, it is necessary to extract the low-frequency component region by, for example, performing fast Fourier transform for an image signal obtained by capturing an image. When an image capturing apparatus shoots a still image, an image capturing circuit and correction circuit perform luminance correction for the still image to be recorded, according to the following procedure.
1. Develop the captured image signal
2. Convert the developed image into a luminance image
3. Extract a low-frequency component region from the luminance image
4. Decide a target luminance correction amount based on the luminance level of the low-frequency component region
5. Perform luminance correction for the low-frequency component region of the developed image with the decided target luminance correction amount
6. Record the image having undergone the luminance correction
If, however, successively captured and obtained images are sequentially input to the correction circuit when, for example, a moving image is shot or live view is active, it is necessary to sequentially perform luminance correction for the input images and output them. That is, timing constraints are imposed on processing from a shooting instruction to a recording operation unlike a still image, that is, a case in which other images are input to the correction circuit.
In shooting a moving image, for example, it is necessary to execute, in parallel, in each frame, correction amount decision processing of deciding a target luminance correction amount based on a captured and obtained image, and recording processing of recording a frame image by performing luminance correction with the target luminance correction amount for a low-frequency component region of the captured and obtained image. In this case, the processes need to be individually performed in different lines at the same time. It is, however, not realistic to provide, in each line, a circuit for extracting a low-frequency component region from a luminance image, since this increases the circuit scale, cost, or power consumption. Considering the above-described problem when luminance correction is performed for a high-frequency component region, it is necessary to provide a circuit for extracting a low-frequency component region on the recording processing line side.
Although it is also possible to provide a circuit for extracting a low-frequency component region on the correction amount decision processing line side, and holding information indicating a low-frequency component region used for the correction amount decision processing and then using it for the recording processing, in the following problem arises in that case. When the correction amount decision processing and the recording processing are simultaneously executed, it is impossible to perform correction processing after a target luminance correction amount is decided unlike a still image because of the timing constraints. The decided target luminance correction amount, therefore, can only be used for the next frame and subsequent frames. That is, if an object is expected to move like a moving image, a low-frequency component region may change between a frame used to decide the correction amount and that to undergo luminance correction when performing luminance correction using the target luminance correction amount. Therefore, luminance correction may be performed for a high-frequency component region, thereby causing degradation in image quality, as described above.
In an arrangement in which the circuit for extracting a low-frequency component region is provided on the recording processing line side, when successively captured and obtained, images subsequently undergo luminance correction, in which a target luminance correction amount is decided based on the luminance levels of pixels which include those in a high-frequency component region. If luminance correction is performed using the thus decided target luminance correction amount for a low-frequency component region of the developed image, the low-frequency component region may not be appropriately corrected; for example, the region may be overcorrected. Thus, a high-quality corrected image may not be output.